Skip to content

Add tokenizer and chat-template provenance gates#2238

Open
Errordog2 wants to merge 1 commit into
UnitOneAI:mainfrom
Errordog2:codex/model-tokenizer-template-provenance-gates
Open

Add tokenizer and chat-template provenance gates#2238
Errordog2 wants to merge 1 commit into
UnitOneAI:mainfrom
Errordog2:codex/model-tokenizer-template-provenance-gates

Conversation

@Errordog2

Copy link
Copy Markdown

/claim #2227

Summary

  • Adds tokenizer and chat-template provenance gates to model-supply-chain.
  • Requires pinned revisions and digest evidence for tokenizer/config/template assets alongside weight digests.
  • Adds special-token diff review, prompt roundtrip tests, runtime parity checks, stop-sequence validation, and final-composition safety evaluation.
  • Updates model inventory/output evidence, common pitfalls, and changelog for model-supply-chain v1.0.1.

Why

Issue #2227 notes that tokenizer and chat-template assets can change prompt semantics, role separation, stop sequences, and safety behavior independently of model weights. This PR makes those assets first-class supply-chain inputs that must be pinned, digested, compatibility-tested, and evaluated as part of the final deployed composition.

Validation

  • git diff --check
  • Frontmatter required-field check across skills and roles
  • index.yaml file existence check
  • Prompt-injection phrase scan across skills and roles
  • Targeted rg check for version and new tokenizer/chat-template sections

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant